1. Field of the Invention
The invention concerns techniques for representing object metadata generally and more particularly concerns techniques for representing object metadata in relational database systems.
2. Description of Related Art
An important task of modem computer systems is dealing with multimedia objects, that is, objects that contain representations of different kinds of information. For example, a video object that contains a digital representation of a TV newscast will include at least a video component, representing the moving image part of the program, and an audio component, representing the sound part of the program, and may also contain a closed-caption text component.
One of the many problems that arise for a computer system in dealing with multimedia objects is finding things within the multimedia object. Of course, the computer system can always begin at one end of the multimedia object and search until it finds what it is looking for, but with objects of any size, this technique simply takes too long, just as takes too long for humans to find things in books that way.
With books, the problem has been solved by adding subdivisions and using the subdivisions to find things. For example, the Christian Bible is divided into Old and New Testaments, each testament is divided into books, books are divided into chapters, and the chapters are divided into verses. In some editions of the Bible, the words of Jesus are printed in a different color from the rest of the text. A given location in the Bible is traditionally specified by book, chapter, and verse, for example, Genesis 1:1. Additionally, when the Bible is printed in book form, the pages of the book are numbered, so that one can speed the search still further by stating that the verse is on a given page of a given edition of the Bible. Various kinds of indexes make use of these subdivisions. The Bible will typically include a table of contents that indicates the page upon which each book starts and many Bibles have concordances that relate a word or term to the verses that contain or are relevant to the word or term.
The books, chapters, verses, colored text, and page numbers of the Bible are examples of metadata, that is, data that is not part of the content of the Bible, but rather serves to make it easier to locate things in the Bible. Most writings include at least some metadata, if only section headings and page numbers, and many kinds of specialized metadata have been developed for various kinds of written works.
Metadata may also be used to find things in multimedia objects. FIG. 1 shows how complex such objects may be. FIG. 1 is a representation of the various kinds of information contained in a digitized video program 101. The video program 101 is made up of a sequence of video frames 103. Within the video frames 103 are a number of different strata (105-117) of information. Some of this information is the actual content of the video; some is metadata. A stratum may further be divided into segments. One example of such segments is video clips; the video clip stratum is shown at 117; an example segment is shown at 121. Another example of a segment is a portion of speech stratum 113 containing one speaker, as shown at 123. Continuing with the other strata and beginning with keyframes stratum 105, to reduce the number of bits required to represent the video images, a number of different kinds of frames are used. Only keyframes 105 contain complete images; the other kinds of frames represent an image as changes with regard to a keyframe. CC stratum 107 is made up of closed-caption texts 107 summarizing the contents of the images and audio, so that people who cannot understand the audio portion can read the captions and determine what is going on.
Stratum 109 is annotations, for example indications of copyright ownership of various portions of the video program. The audio portion of the video program may include two strata, one, speech, 113, which actually represents music or speech, and another, audio classes 119, which indicates what the current audio track is representing. Where a speaker is speaking, there may be a speaker identification stratum 115 identifying a segment of the audio. Clips stratum 117, finally, indicates segments of the video which may be used as clips.
As is apparent from the foregoing, some of the strata, for example clips 117, are themselves metadata with regard to the video itself; each of the strata, however, may have its own metadata. Many different kinds of metadata have been used with multimedia objects; an emerging standard in the area is the Dublin Core, described in Stuart Weibel, Jean Godby, Eric Miller, and Ron Daniel, OCLC/NSA Metadata Workshop Report, which was published in 1995 and could be found in March, 2000 on the Internet at www.oclc.org:5046/oclc/research/conferences/metadata/dublin_core_report.html.
Dealing with multimedia objects has been a particular problem for relational database systems. First, multimedia objects are too large to be easily incorporated into relational database tables. Second, the data in them is different from that usually stored in database tables. Third, different multimedia objects may have different locations, for example in storage internal to the database system, in storage internal to the computer system the database system is executing on, or in a Web server, and the ability of the database system to deal with the object consequently cannot depend on where the object is located. These problems have been solved in the object-oriented relational database system Oracle 8(trademark), manufactured by Oracle Corporation. Their solution is described in Oracle8i interMedia Audio, Image, and Video User""s Guide and Reference, Part No. A67299-01, Release 8.1.5, Copyright(copyright) 1999, Oracle Corporation, which is incorporated by reference into the present patent application.
What has remained to be solved are the problems which the metadata associated with a multi-media object present for a relational database system. It has of course always been possible for an application program to use the relational database system to locate a multi-media object and then manipulate the metadata in the multi-media object. A first problem with this solution is that the application cannot use the database system""s indexing and querying capabilities to deal with the metadata; a second problem is that each application must include custom code to deal with the metadata; a third is that in order for the application program to deal with the metadata, the entire object containing the metadata must be sent from the database system to the application program.
It is of course also possible for the manufacturer of a relational database to modify a relational database system to deal with particular kinds of metadata. This has been done in the DB2 Digital Library, version 2.4, manufactured by International Business Machines Corporation, where the relational database system has been modified to deal with the metadata used in multimedia workflow management. Information about the DB2 Digital Library may be found at www.software.ibm.com/dglib. The difficulty with this situation is that in a rapidly-evolving environment like that of multimedia data, the database system cannot be modified rapidly enough to keep up with the changes in the metadata.
Another approach to dealing with multimedia data, including metadata, is to permit the user to program his or her special purpose modules for the database system. This approach is exemplified by the Datablade(copyright) technology available to users of database systems produced by Informix Corporation of Menlo Park, Calif., as described in the Informix DataBlade Technology data sheet available in April, 2000 at www.informix.com/informix/products/integration/datablade/datablade_ds.htm. While this keeps the processing of the metadata and the objects containing them in the database system and does not require the user to wait for the database system manufacturer to modify the database system, it still requires that special module be built for each new kind of object to be handled by the database system. It is up to special module for a class of object to handle the object""s metadata.
What is needed is techniques for dealing with metadata within the database system that are at the same time built into the database system, so that there is no need to build special modules, and flexible enough to handle all present and future kinds of metadata. It is an object of the invention described herein to provide such techniques for handling metadata.
The object of the invention is attained by providing a generic representation for metadata. The generic representation represents an item of metadata by specifying the stratum in the multimedia object to which the item of metadata-belongs, the portion of the stratum to which the item of metadata belongs, and the metadata item""s value. The specification of the stratum, portion, and the metadata item""s value are all done using data types that are built into the database system, and the values used in the specification may be set at runtime. The generic representation is part of the database, and consequently, metadata may be located by or used in queries like any other data in the database.
The specifier in the generic representation that specifies the portion of the stratum is termed the extent specifier. Only a relatively small number of kinds of extents are required to describe the portion of the stratum that a metadata item may apply to. The extent specifier may specify a period of time in the stratum, for example for audio data, a region of the stratum, for example for image data, or may specify a tag for a region in the stratum such as a page in a document. In addition the extent specifier may specify that the metadata item is global, i.e., applies to the entire stratum. The extent specifier for a segment contains two parts: an extent type specifier and an extent value specifier. The metadata item""s value is similarly specified by a value type specifier and a value specifier.
The generic representation for metadata is employed in an object relational database system in which a multimedia object is represented in the database system""s tables by a first object and the generic representation of the metadata for the multimedia object is represented by a second object that is contained in the first object. Methods for the generic representation include methods that manipulate metadata values and methods that return parts of multimedia objects specified by metadata values. Parameters for the methods allow specification of whether the method is to apply to a metadata value for a particular portion of a stratum, to a global metadata value, or a metadata value for any portion of the stratum. Methods are also provided for bulk loading of metadata values into the database.
Other objects and advantages will be apparent to those skilled in the arts to which the invention pertains upon perusal of the following Detailed Description and drawing, wherein: